MS&E 336 Lecture 4: Stochastic games

i Ai(x), a stage payoff Qi(a;x). 4. For each state x and action vector a ∈ ∏ i Ai(x), a transition probability P(x |x,a) that is a distribution on the state space X . 5. A discount factor δ, 0 < δ < 1. 6. An initial state x. Play proceeds as follows. The game starts in state x. At each stage t, all players simultaneously choose (possibly mixed) actions ai, with possible pure actions given by the set Ai(x ). The stage payoffs Qi(a;x) are realized, and the next state is chosen according to P(·|x,a). All players observe the entire past history of play before choosing their actions at stage t. (This is the simplest assumption; versions with partial monitoring have also been studied.) As usual, let si denote a strategy for player i in this dynamic game; it is a mapping from histories (including states and actions) to actions. (After any history leading to state x, player i’s strategy must choose an action in Ai(x).) Given strategies s1, . . . , sN , the expected discounted payoff of player i starting from state x is: